(12) INTERNATIONAL APPLICATION PUBLISHED UNDER THE PATENT COOPERATION TREATY (PCT) 



(19) World Intellectual Property Organization 

International Bureau 

(43) International Publication Date 
28 November 2002 (28.11.2002) 




PCT 



(10) International Publication Number 

WO 02/095680 Al 



(51) International Patent Classification 7 : G06T 1/00 

(21) International Application Number: PCT/IB 02/0 1841 

(22) International Filing Date: 21 May 2002 (21.05.2002) 

(25) Filing Language: English 

(26) Publication Language: English 



(30) Priority Data: 

01201974.1 



23 May 2001 (23.05.2001) EP 



(71) Applicant (for all designated States except US): KONIN- 
KLIJKE PHILIPS ELECTRONICS N.V. [NL/NL]; 
Groenewoudseweg 1, NL-5621 BA Eindhoven (NL). 

(72) Inventors; and 

(75) Inventors/Applicants (for US only): WILINSKI, Piotr 



[PL/NL]; Prof. Holstlaan 6, NL-5656 AA Eindhoven (NL). 
ERNST, Fabian, E. [DE/NL]; Prof. Holstlaan 6, NL-5656 
AA Eindhoven (NL). 

(74) Agent: GROENENDAAL, Antonius, W., NL; Interna 
tionaal Octrooibureau B.V., Prof. Holstlaan 6, NL-5656 
AA Eindhoven (NL). 

(81) Designated States (national): CN, JP, KR, US. 

(84) Designated States (regional): European patent (AT, BE, 
CH, CY, DE, DK, ES, FT, FR, GB, GR, IE, IT, LU, MC, 
NL, PT, SE, TR). 

Published: 

— with international search report 

For two-letter codes and other abbreviations, refer to the "Guid- 
ance Notes on Codes and Abbreviations" appearing at the begin- 
ning of each regular issue of the PCT Gazette. 



(54) Title: DEPTH MAP COMPUTATION 




(57) Abstract: Method for computation of a depth map for a digital image (IM) composed of pixels, with the steps of receiving 
digital image data (700), receiving singularity data (rec-inf) for the digital image (IM), receiving depth value data (dd) for segments 
of the digital image (IM), segmenting the digital image (IM) into segments based on the singularity data (rec-inf) by assigning each 
pixel of the digital image (IM) to a segment, assigning to each segment corresponding depth value data from the received depth value 
data (dd), and constructing a depth map (800) by assigning to each respective pixel the corresponding depth value data (dd) of the 
segment to which the respective pixel is assigned. 



WO 02/095680 

Depth map computation 



PCT/IB02/01841 



The invention relates to a method for computation of a depth map for a digital 
image. The invention further relates to a method for compressing digital image information, a 
decoder and an encoder for digital image information. 

In digital image processing, depth information related to elements of an image 
5 is relevant in a number of applications such as for example 3D TV. The depth information is 
needed to process the video information as to reconstruct 3D images to be shown on the TV 
screen. Depth information can be extracted out of the subsequent images forming video 
material in real time in the 3D TV or in a set-top box. This approach has the disadvantage 
that it is rather costly because of the considerable calculation resources that are required. 

10 It is an object of the invention to provide for a more efficient method for the 

computation of a depth map of a digital image. The invention provides a method according to 
claim 1 . Depth map reconstruction data is provided for an image by singularity data and 
depth value data. Singularity data is information relating to singularities or discontinuities in 
the image that are used for segmenting the image; depth value data is information with 

15 respect to the depth assigned to a segment, relative to other segments in the image. 
Determination of depths of segments in images as such is known in the art. With this 
information, which does not take up a lot of space or bandwidth, a full depth map can be 
reconstructed by the receiver of the information. To do this a method according to claim 5 is 
provided in which the receiver segments the image using a segmentation method, based on 

20 the supplied singularity data. Starting from the singularities, forming the segments can be 
performed with relatively small calculation resources. Finally the depth values are assigned 
to each segment. 

By determining the segments by means of signed distance transform, the 
method according to the invention is in particular advantageous as singularity data defining 
25 the segments, are generated during segmentation in the form of seed points. Therefore, no 
additional calculations are required to obtain relevant singularity data. The seed points 
themselves contain enough information to generate a segmentation of an image with 
relatively small calculation resources. The storage space or bandwidth required by the seed 
point for an image is relatively small, in particular when the seed points are defined as edge 
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positions composed of a grid point, and an up/down and left/right indicator for the grid point. 
With each segment generated by the seed points, depth information has to be included, which 
information again requires only little storage or bandwidth. 

Particularly advantageous elaborations of the invention are set forth in the 
5 dependent claims. Further objects, elaborations, modifications, effects and details of the 

invention appear from the following description, in which reference is made to the drawings, 
in which 

Figure 1 shows hard edges in an image, 

Figure 2 shows seeds associated with the hard edges of Figure 1, 
10 Figure 3 shows fronts expanding from the seeds of Figure 2, 

Figure 4 shows image segments determined by segmentation according to an 
embodiment of the invention, 

Figure 5 shows the image segments of Figure 4 after post-processing, 
Figure 6 shows a segment boundary with seed points, 

Figure 7 is a flow chart of an encoding method according to an embodiment of 
Figure 8 is an illustration of an edge position, 

Figure 9 is a flow chart of a decoding method according to an embodiment of 

Figure 10 is a television set with an encoder according to an embodiment of 

Figure 1 1 is television set with a set top box according to an embodiment of 

Figure 12 is an encoder according to an embodiment of the invention, 
Figure 13 is a decoder according to an embodiment of the invention, 
Figure 14 is a transmitter according to an embodiment of the invention, and 
Figure 15 is a storage medium provided with data files stored on it according 
to an embodiment of the invention. 

In the following an example of a method according to the invention will be 
30 described. In the example, a digital image M composed of pixels will be described. Such an 
image can for example be an image comprised in a video data stream; although the example 
will deal with a single image, the invention is in particular suited to be used for multiple, 
subsequent images. 



15 

the invention, 



the invention, 

20 

the invention, 



the invention, 



25 
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According to the invention, the processing of the image comprises dividing the 
image into segments (segmentation). An efficient method of dividing the image into 
segments, called quasi-segmentation, is described below. 

In the following examples of the quasi-segrnentation use will be made of 
5 segmenting a digital image into separate regions. The digital image is composed of image 
pixels. Segments to be formed in the image are bounded by borders or border lines; pixels 
within the borders of a segment belong to that segment. Therefore the determination of the 
borders leads to the determination of the segments. 

To obtain borders or at least fragments of borders, the digital image is 
10 processed to find edges in the image using an edge detection process, in which image features 
are analyzed. Edges that are detected result from image features, and have therefore a high 
probability to be a boundary between image objects. The edges detected by the edge 
detection process are used as fragments of the borders between the segments to be 
determined. These border fragments resulting directly from image information are called hard 
1 5 border fragments or high probability border fragments. 

In Figure 1 hard border fragments detected in the image with an edge detection 
method are shown in image 10 which has the same size as the digital image. Three hard 
border fragments have been detected, border fragments a, b, and c respectively. Note that 
border fragment b is incident at border fragment a; this topography is known as a bifurcation. 
20 Edge detection methods per se are well known in the art. In this example the 

hard border fragments are determined by high contrast borders, which are a good indication 
of a border between image elements. Other criteria for border fragments between image 
elements can be used such as color, luminance, or texture. 

The hard border fragments a, b, c bound part of a segment; the borders of the 
25 segments are not complete however. The other sections of the borders have to be established. 
The other border sections are determined by the distance to the closest hard border section. 
To obtain the other border sections, sides of the border fragments a, b, and, c are defined and 
uniquely labeled. As shown in Figure 2, border section b has a first side IV and a second side 
V, and border section c has therefore a first side VI and a second side VII. Border section a 
30 has a first side III; the other side of border section a is divided in two sections by border 

section b in the position where border b intersects border section a. The respective sections 
are sides I and II of border section a. 

To obtain the other boundaries, the sides I-VII are expanded in a direction 
away from the border fragment from which they originate, the respective expansion 
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directions being indicated by arrows F-VIF in Figure 3. Preferably, the direction of 
expansion is essentially perpendicular to the previous front. A number of expanding fronts, 
labeled Ia/b/c - Vlla/b/c, respectively, have been indicated in Figure 3, wherein the suffix -a 
denotes fronts close to the original edge, and the suffixes -b and -c respectively denote 
5 subsequent fronts further from the original border fragment. In fact each front is the locus of 
point having the same distance to the closest border fragment. Where the expanding fronts 
meet neighboring expanding fronts, a border fragment is formed, as indicated by the hatched 
lines in Figure 4. These border fragments are called soft border fragments as they do not 
derive directly from information of the image. The soft border sections are essentially 

10 contiguous to the end sections of hard border sections. Non contiguous soft border sections 
can however occur, e.g. when hard border sections extend up to the edge of the image. The 
probability that a soft border is a part of a border of a segment is lower than that of the 
aforementioned hard borders. After full expansion of the fronts up to the edge of the image, 
segments are defined as shown in Figure 4, indicated by capitals A-E. The soft boundaries 

1 5 are labeled by the two segments they divide. As a result the complete image has been divided 
in segments A-E, wherein each segment is bound at least partially by a hard border and 
further by soft borders or the image edge. Subsequently, the obtained segmentation can be 
scanned for oversegmented regions, that logically form a single segment. In this example the 
borders between segments B1-B2 and C1-C2 are redundant, as a result of oversegmention, 

20 caused by the bifurcation of border a and b. After detection of such oversegmentation, the 
segments Bl, B2 and CI, C2 can be merged. 

Consequently, image pixels can uniquely be assigned to a segment, bound by 
the hard and soft borders sections, as established in the above explained manner. Note that 
the segments consist of groups of pixels that share the same closest side of a hard border 

25 fragment. 

The segmentation as obtained with this method is called a quasi segmentation, 
wherein some sections of the boundaries of the segments are less strict defined, with a lower 
degree of certainty (the above described soft border sections). This quasi segmentation has 
the advantage that it results in a segmentation which is accurate in sections of the borders 
30 where the segmentation can be determined easily, and less accurate in sections where 

determination is more difficult. This results in significantly decreased calculation costs, and 
increased calculation speeds. The quasi segments can for example be used in matching of 
segments in subsequent images. 
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In the following an implementation of quasi segmentation will be described. 
The digital image to be segmented in this example is a discrete image IM (x, y) with pixels 
(x, y) of a resolution NxM, wherein N and M are integers. A binary picture I(x,y) with pixels 
(x, y) of resolution NxM is defined; the binary picture I(x,y) is used for the determination of 
5 the segments of the image IM as hereinafter described. Also defined are an array d(x,y), 
called the distance array with size NxM and an array b(x, y), called the item buffer, again 
with size NxM. In the distance array d(x,y) for every pixel (x,y) the distance to a closest seed 
(as defined below) is stored; the determination of this distance will be explained in the 
following. In the item buffer array b(x,y) is stored for every pixel (x,y) the identity of the 

10 closest seed or border fragment; in the following the determination of the closest seed or 
border will be explained. 

First the digital image IM is processed with an edge detector to determine well 
defined edges; this step is similar to the detection of hard border fragments mentioned before. 
By way of example the known Marr-Hildreth method is used in this embodiment, as 

15 described by E. C. Hildreth, in "The detection of intensity changes by computer and 
biological vision systems" published in 1983 in Computer vision, graphics and Image 
processing, pag. 22:1-27. The Marr-Hildreth algorithm uses zero crossings of a Laplacian of 
Gaussian (LoG) operator to detect border fragments. 

The Marr-Hildreth method detects zero crossings of the LoG in between two 

20 pixels of the discrete image IM, which are considered as points on hard border fragments as 
in the first embodiment. In Figure 6 a section of an image matrix is shown, with the 
intersections of the grid indicating the location of the pixels. The line 305 indicates the zero 
crossings, indicated by the asterisks (*) 310, detected by means of the LoG operator. The 
hard borders found in the image by the LoG zero crossings detection are mostly extended 

25 contiguous sequences of inter pixel positions. With each zero crossing, that lies between two 
pixels, two seed pixels are associated on either side of the crossing; the border 305 passes in 
between the two seed pixels. In this embodiment a seed consists of seed pixels, wherein seed 
pixels are the pixels of the image that are closest to the hard border sections. The seeds form 
an approximation of the border sections within the digital image pixel array; as the seeds fit 

30 within the pixel array, subsequent calculations can be performed easily. Other methods of 
determining seeds on basis of found hard border sections can be used. The pairs of seed 
pixels opposite the border 310 are indicated by the circles 320 and the black dots 330 in 
Figure 6. 
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Seed pixels are defined all along the detected hard border 305, giving rise to 
two-pixel wide double chains. Each chain of seed pixels along a side of the border (i.e. each 
one-pixel wide half of the double chain) is regarded as a seed, and accordingly indicated by a 
unique identifier. As the hard border in this example is defined by a zero crossing of the LoG 
5 operator, the value of LoG is positive on one side of the border and negative on the other 
side. Identification of the different sides of the border can be achieved according to the 
invention by using the sign of the LoG operator. This is advantageous as the LoG operator 
has already been calculated during the process. Because of the use of the LoG operator the 
method of segmentation can also be referred to as Signed Distance Transform. 

10 As a result of the LoG based edge detection the seed pixels essentially form 

chains; however, seeds can be arbitrarily shaped clusters of edge pixels, in particular seeds 
having a width of more than a single pixel. 

In the item buffer b(x,y), the value corresponding with the position of a seed 
point is given the value of the unique seed identifier. Initially, all other pixels, which are not 

1 5 seed points, do not have a seed identifier number in the item buffer b(x,y), but are given a 
value that does not correspond to a seed identifier number. 

For each pixel of the image IM(x,y) which is found to be a seed pixel, the 
pixel with the corresponding coordinates (x,y) in the binary image I is given the value 1 . All 
other pixels in the image I are given the value 0. 

20 By means of for example linear interpolation of the values in the LoG-filtered 

image, an estimation can be made for the sub-pixel distances between the actual zero 
crossing 310 and the respective pair of seed pixels 320, 330. As shown in Figure 6 for the 
pair of pixels at the farthest right hand side, the respective distances are dl and d2, wherein 
dl 4- d2 = 1, wherein the grid size for the pixel distance is the unit distance 1. The respective 

25 values for dl and d2 are assigned to d(x,y) for the respective seed pixels. The distance array d 
is further initialized by assigning a distance corresponding to infinity within the distance 
system used to pixel positions not on a seed. 

A distance transform gives, for every pixel (x, y) the shortest distance d(x,y) to 
the nearest seed point. Any suitable definition for the distance can be used, such as the 

30 Euclidean, "city block" or "Manhattan" distance. Methods for calculating the distance to the 
nearest seed point for each pixel are well known in the art, and implementing the invention 
any suitable method can be used. By way of example an algorithm may be used for the 
computation of the distance transform as described by G. Borgefors in "Distance transforms 
in arbitrary dimensions", published in 1984 in Computer vision, graphics and Image 
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processing, pag. 27:321-345, and in particular the disclosed method for the two dimensional 
situation. 

This algorithm is based on two passes over all pixels in the image I(x,y), 
resulting in values for d(x,y) indicating the distance to the closest seed. The values for d(x,y) 
5 are initialized as mentioned before. In the first pass, from the upper left to lower right of 
image I, the value d(x,y) is set equal to the minimum of itself and each of its neighbors plus 
the distance to get to that neighbor. In a second pass, the same procedure is followed while 
the pixels are scanned from the lower right to upper left of the image I. After these two 
passes, all d(x,y) have their correct values, representing the closest distance to the nearest 
10 seed point. 

During the two passes where the d(x,y) distance array is filled with the correct 
values, the item buffer b(x,y) is updated with the identification of the closest seed for each of 
the pixels (x,y). After the distance transformation, the item buffer b(x,y) has for each pixel 
(x,y) the value associated with the closest seed. This results in the digital image being 

1 5 segmented; the segments are formed by pixels (x,y) with identical values b(x,y). 

In the second example the distances that are computed further in the distance 
transform algorithm are non-integer values, for example a real number, because of the linear 
interpolation of the starting values of d(x,y). When comparing for a pixel (x,y) the real- 
valued d(x,y) values representing the shortest distances to two different seeds, the chance that 

20 both distances are different is very large. This allows a unique identification of every pixel as 
belonging to one single segment. If distances were to be measured in integer values, arbitrary 
choices would have to be made for each of many pixels that would have the same distance to 
two seeds. This would lead to increased raggedness (and therefore reduced accuracy) of the 
border, but with lower calculation power requirements. 

25 In Figure 7 a flow chart for a method for encoding a digital image according to 

the invention is shown. 

The first step of the processing of the digital image M is the segmentation 100 
of the image, for example using the above described method of quasi-segmentation. In short, 
the image is scanned for singularities as required in quasi-segmentation, in particular 

30 luminosity edges. Pixels surrounding the found edges are used to determine seed points, 

making up seeds. The seeds are expanded to form segments. As shown above the result of 
this segmentation is that each pixel of the image is assigned to a segment, a segment 
therefore being a group of pixels. The results are the locations of the seeds within the image, 
and a filled out item buffer b. 
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In a subsequent step 200 depth values for each segment and therefore each 
pixel in the item buffer are determined, yielding a depth map dm. Determination of depth 
values per se is known from the art, and according to the invention any suitable method can 
be used. 

5 In step 300 the information with respect to the depth values determined for the 

image is compressed. This is done by composing depth reconstruction information for the 
digital image, based on the information resulting from the segmentation and depth analysis. 
On basis of the reconstruction information a depth map for the image can be reconstructed. 

To achieve this, in the reconstruction information only the edge positions of 

1 0 the segments 3 1 0 are included together with the depth value data 320 of the segment that is 
engendered by the segment. The receiver can use this reconstruction information to 
regenerate a depth map for the digital image, using the above described segmentation 
method, starting with the edges provided. It is noted that in the quasi-segmentation the step 
that requires the most calculation resources is the determination of the singularities. Once the 

1 5 singularities are known, forming the segments can be performed with relatively small 
calculation resources. 

The edge information can be coded as follows. In Figure 8 a section of a grid 
of an image is shown. Parts of three segments Dl, D2, and D3 are shown, separated by two 
edges el, e2. For storing the edge information an edge position needs 

20 • the coordinate of a grid point (x, y) to which the edge is attributed, 

• information of the presence or absence of an edge crossing the grid at the upper side of 
grid point (x, y), 

• information of the presence or absence of an edge crossing the grid at the right side of 
grid point (x, y), and 

25 • the depth values of the engendered segments. 

For the situation of Figure 8, the edge crossings are respectively at the upper 
and right side of the grid point (x, y), indicated accordingly with a + sign. For the definition 
of the item buffer the precise location of the zero crossings on the edge between the grid 
points (dl, and d2 shown above) is not required. Therefore the presence information can 
30 sufficiently be represented by a binary or Boolean parameter. 

Alternatively, the edge information can be coded using the information with 
respect to the seeds found in the segmentation process. In this case the data to be transmitted 
comprises: 

• the seed pixel coordinates, 
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• the respective seed numbers, and 

• a table attributing depth values to seed numbers. 

The number of seed pixel coordinates is roughly twice that of the number of 
edge positions; therefore transmitting edge information through seed pixel coordinates 
5 requires a larger data transmission. The reconstruction of the segments is slightly faster, 
because there is no need to reconstruct the seed points. 

Subsequently, in a following step 400, shown in phantom in Figure 7, the 
digital image is transmitted to a receiver, together with the reconstruction information. 
Depending on the transmission protocol used, the reconstruction information can be 

1 0 transferred using a parallel communication channel, for example as provided in MPEG. 
Alternatively, the reconstruction information can be stored on a data carrier, such as for 
example a Digital Versatile Disk, CD and CD-ROM, shown in phantom in Figure 7 as step 
500, preferably together with the digital image information, using a suitable storage method, 
such as MPEG. The data determined in step 300 is consecutively output, shown in phantom 

15 in Figure 7 as steps 400 and 500. 

According to the invention an encoder device 600 for compressing digital 
image information is provided, as shown in Figure 12. The device 600 comprises an input 
section 610 for receiving digital images composed of pixels, a processing unit 620 for 
segmenting a digital image based on singularities in the digital image by assigning each pixel 

20 of the digital image to a segment, and for determining of depth value data for each segment 
of the image, and an output means 630 for outputting depth reconstruction information for 
the digital image, comprising said singularity data and depth value data. Preferably the 
processing unit 620 is provided with a computer program for performing the steps 100, 200, 
300 of the encoding method described above. The invention is however not limited to this 

25 implementation. Other ways of implementation can be used, for example using dedicated 
hardware, such as a chip. 

In Figure 14 a transmitter 950 according to the invention is shown, provided 
with an encoder 600 as described above. The transmitter is further provided with an input 
section 955 for receiving image information and an output section 965, embodied in this 

30 example as a send device. The send device 965 is adapted to generate a output signal, for 

example a digital bit stream signal or a signal suitable for broadcasting. The signal generated 
represents a digital images and comprises singularity data for the digital image, and depth 
value data for segments of the digital image. 
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The information transmitted or read from a data carrier as produced by the 
above described method is processed by a receiver, as shown in a flow chart in Figure 9. The 
receiver receives (step 700) the image information IM and the reconstruction information rec- 
inf, the reconstruction information being formed by the singularity information and the depth 
5 values. Using the reconstruction information rec-inf, the segmentation of each image of the 
image information is reconstructed, and the depth map for the image formed (step 800) by 
using the depth value data dd contained in the reconstruction information. The depth map can 
subsequently be used for displaying the image information as shown in phantom as step 850. 

The method of encoding the information encoded according to the above 

10 mentioned steps 100, 200, 300 comprises receiving digital image data, receiving singularity 
data and depth value data for segments of the digital image. As shown before, the singularity 
data forms the basis for finding a segmentation. Two examples are shown, the first one 
comprising singularity data in the form of edge information and the second one comprising 
singularity data in the form of seed information. According to the above mentioned method 

15 of segmentation, using either edges or seeds, a segmentation and corresponding item buffer 
of the image can be calculated. Consequently, a depth map can be constructed by matching 
the depth information provided with the received information to the item buffer. This results 
in a depth map in which each pixel is provided with a depth value. Forming segments starting 
from singularities, such as edges or seeds, is a relatively easy operation which does not 

20 require large calculation resources. 

According to the invention an decoder device 900 for computation of a depth 
map for a digital image composed of pixels is provided as shown in Figure 13. The decoder 
900 comprises an input section 930 for receiving digital image data, singularity data for said 
digital image, and depth value data for segments of said digital image, processing section 920 

25 for segmenting a received digital image into segments using said singularity data by 

assigning each pixel of said digital image to a segment, and for constructing a depth map by 
assigning to each respective pixel the received depth value data of the segment to which the 
respective pixel is assigned, and an output section 910 for outputting said depth map. 
Preferably the processing unit 920 is provided with a computer program for performing the 

30 steps 700, 800, 850 of the encoding method described above. 

In Figure 10 a television 950 is shown, provided with a decoder 900, the 
output section of which decoder 900 is connected to a display driver unit 960 for a television 
display 955. In Figure 1 1 a television 980 is shown, provided with a television display 955 
and a display driver unit 960. The television is connected to a decoder 900 which is 
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implemented as a set top box, A video signal comprising reconstruction information as 
described above can be fed to the television 950 directly, after which the decoder 900 
processes the information so the driver 960 can display the images on the display 955. 
Accordingly, a video signal comprising reconstruction information as described above can be 
5 fed to the set top box shown in Figure 1 1 , after which the decoder 900 processes the 

information and feeds it to the television 980 so that the driver 960 can display the images on 
the display 955. 

The steps of the method of decoding and encoding according the invention as 
described above, can be performed by program code portion executed on a computer system. 

1 0 The invention therefore further relates to a computer program with code portions that when 
executed on a computer system perform the steps of encoding and/or decoding. Such a 
program can be stored in any suitable way, for example in a memory or on an information 
carrier, such as a CD-ROM or floppy disk 980, as shown in Figure 15. 

It should be noted that the above-mentioned embodiments illustrate rather than 

15 limit the invention, and that those skilled in the art will be able to design many alternative 
embodiments without departing from the scope of the appended claims. In the claims, any 
reference signs placed between parentheses shall not be construed as limiting the claim. The 
word 'comprising' does not exclude the presence of other elements or steps than those listed 
in a claim. The invention can be implemented by means of hardware comprising several 

20 distinct elements, and by means of a suitably programmed computer. In a device claim 

enumerating several means, several of these means can be embodied by one and the same 
item of hardware. The mere fact that certain measures are recited in mutually different 
dependent claims does not indicate that a combination of these measures cannot be used to 
advantage. 
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CLAIMS: 



1 . Method for computation of a depth map for a digital image (IM, M) composed 
of pixels, comprising 

receiving digital image data, 
characterized by 

5 receiving singularity data (rec-inf) for said digital image (IM, M), 

receiving depth value data (dd) for segments of said digital image (IM, M), 
segmenting said digital image (M, IM) into segments based on said singularity 
data (rec-inf) by assigning each pixel of said digital image (IM, M) to a segment, 

assigning to each segment corresponding depth value data from said received 
10 depth value data (dd), and 

constructing a depth map (dm) by assigning to each respective pixel the 
corresponding depth value data (dd) of the segment to which the respective pixel is assigned. 

2. Method according to claim 1, farther comprising 

15 segmenting said digital image (IM, M) by means of signed distance transform. 

3. Method according to claim 2, wherein segmenting said digital image (IM, M) 
by means of signed distance transform further comprises 

determining seeds associated with said singularity data, 
20 expanding the found seeds to fill an item buffer (b), and 

constructing a depth map (dm) by attributing corresponding received depth 
value data (dd) to the item buffer (b). 

4. Method according to claim 2, wherein segmenting said digital image (IM, M) 
25 by means of signed distance transform further comprises expanding seeds and seed numbers 

included in said singularity data to fill an item buffer (b), and 

constructing a depth map (dm) by attributing corresponding received depth 
value data (dd) to the seed numbers in the item buffer (b). 
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5. Method for compressing digital image information comprising 
determining singularities in a digital image (IM, M) composed of pixels, 
characterized by 

segmenting a digital image (IM, M) based on said determined singularities by 
5 assigning each pixel of said digital image (IM 5 M) to a segment, 

determining of depth value data (dd) for each segment of said image (IM, M), 
determining singularity data for said digital image (IM, M), and 
composing depth reconstruction information (rec-inf) for said digital image 
(IM, M), comprising said singularity data and depth value data (dd). 

10 

6. Method according to claim 5, further comprising segmenting said digital 
image (IM, M) by means of signed distance transform. 

7. Method according to claim 6, wherein segmenting said digital image (IM, M) 
15 by means of signed distance transform further comprises 

finding seeds associated with said singularity data, 
expanding the found seeds to fill an item buffer (b), and 
constructing a depth map (dm) by attributing corresponding depth value data 
to the item buffer (b). 

20 

8. Method according to any one of claims 5-7, further comprising 
determining edges as singularities in said digital image, 
determining as singularity data edge positions comprising 

- a grid point, 

25 - an up/down indicator associated with said grid point, and 

- an left/right indicator associated with said grid point. 

9. Method according to claim 8, wherein a Boolean parameter is used for the 
respective up/down indicator and left/right indicator. 

30 

1 0. Method according to claim 7, further comprising 
determining seed points as singularities in said digital image, 

determining as singularity data seeds comprising 

- seed pixel coordinates, 
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associated seed number, 

depth value data associated with said seed number. 
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1 1 . Method according to claim 5, further comprising 

5 transmitting said digital image and said depth reconstruction (rec-inf) 

information to a receiver. 



12. Method according to claim 5, further comprising 

storing said digital image (IM, M) and said depth reconstruction information 
10 (rec-inf) on a data carrier (980). 



13. Decoder device for computation of a depth map (dm) for a digital image (IM, 

M) composed of pixels, comprising 

an input section (610) for receiving digital image data, singularity data for said 
15 digital image (IM, M), and depth value data (dd) for segments of said digital image (IM, M), 
processing section (620) for segmenting a received digital image (IM, M) into 
segments using said singularity data by assigning each pixel of said digital image (IM, M) to 
a segment, and for constructing a depth map (dm) by assigning to each respective pixel the 
corresponding received depth value data (dd) of the segment to which the respective pixel is 
20 assigned, and 

an output section (630) for outputting said depth map (dm). 



14. Encoder device for compressing digital image information comprising 

an input section (610) for receiving digital images (IM, M) composed of 

25 pixels, 

a processing unit (620) for segmenting a digital image (IM, M) based on 
singularities in said digital image (IM, M) by assigning each pixel of said digital image (IM, 
M) to a segment, and for determining of depth value data (dd) for each segment of said image 
(IM, M), and 

30 an output means (630) for outputting depth reconstruction information (rec- 

inf) for said digital image (IM, M), comprising said singularity data and depth value data 
(dd). 
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15. A television provided with a display (955), a display driver (960), and a 
decoder (900) according to claim 13. 

16. A transmitter provided with an encoder (600) according to claim 14, a sending 
5 device (965). 

17. A digital signal representing a digital image comprising singularity data for 
said digital image (IM 5 M), and depth value data for segments of said digital image (IM, M). 

10 18. A data carrier on which a signal as claimed in claim 1 7 has been stored. 

19. Computer program comprising code portions that when executed on a 

computer system perform the steps of claim 1 . 



15 



20. Computer program comprising code portions that when executed on a 

computer system perform the steps of claim 5. 



WO 02/095680 



PCT/IB02/01841 





Fig.2 



WO 02/095680 



PCT/IB02/01841 




WO 02/095680 



PCT/IB02/01841 




WO 02/095680 



PCT/IB02/01841 




WO 02/095680 



PCT/IB02/01841 




Fig.6 



WO 02/095680 



PCT/IB02/01841 




WO 02/095680 



PCT/IB02/01841 




WO 02/095680 



PCT/IB02/01841 



8/9 




-600 



Fig.12 



930 






92 


!0 



900 



910 



Fig.13 



WO 02/095680 



PCT/IB02/01841 




INTERNATIONAL SEARCH REPORT 





In rational Application No 

PCT/IB 02/01841 


A. CLASSIFICATION OF SUBJECT MATTER 

IPC 7 G06T1/00 








According to International Patent Classification (IPC) or to both national classification and IPC 






B. FIELDS SEARCHED 


Minimum documentation searched (classification system followed by classification symbols) 

tpp 7 that 

lit / toUOl 


Documentation searched other than minimum documentation to the extent that such documents are included in the fields searched 


Electronic data base consulted during the international search (name of data base and, where practical, search terms used) 

EPO-Internal , PAJ, INSPEC, WPI Data 


C. DOCUMENTS CONSIDERED TO BE RELEVANT 


Category ° 


Citation of document, with indication, where appropriate, of the relevant passages 


Relevant to claim No. 


X 


MURRAY D ET AL: "Stereo vision based 
mapping and navigation for mobile robots" 
ROBOTICS AND AUTOMATION, 1997. 
PROCEEDINGS., 1997 IEEE INTERNATIONAL 
CONFERENCE ON ALBUQUERQUE, NM, USA 20-25 
APRIL 1997, NEW YORK, NY, USA, IEEE, US, 

20 April 1997 (1997-04-20), pages 
1694-1699, XP010235559 
ISBN: 0-7803-3612-7 

figures 5-10 
page 1696, left-hand column, paragraph 3.2 
-right-hand column, paragraph 4 
page 1697, left-hand column, line 1 - line 
8 

page 1697, right-hand column, paragraph 
5.1 -page 1698, right-hand column, 
paragraph 5.2 




1,5, 
11-20 


Y 




2-4,6-10 






■/— 






| X| Further d° cu ments are listed in the continuation of box C. 


j)( Patent family members are listed in annex. 


° Special categories of cited documents : 

•A" document defining the genera! state of the art which is not 
considered to be of particular relevance 

■E' earlier document but published on or after the international 
filing date 

"L 1 document which may throw doubts on priority claim(s) or 
which is cited to establish the publication date of another 
citation or other special reason (as specified) 

•O" document referring to an oral disclosure, use, exhibition or 
other means 

"P" document published prior to the international filing date but 
later than the priority date claimed 


■T" later document published after the international filing date 
or priority date and not in conflict with the application but 
cited to understand the principle or theory underlying the 
invention 

"X" document of particular relevance; the claimed Invention 
cannot be considered novel or cannot be considered to 
involve an inventive step when the document is taken alone 

"Y" document of particular relevance; the claimed Invention 

cannot be considered to involve an inventive step when the 
document is combined with one or more other such docu- 
ments, such combination being obvious to a person skilled 
in the art. 

document member of the same patent family 


Date of the actual completion of the international search 


Date of mailing of the international search report 


14 August 2002 


28/08/2002 




Name and mailing address of the ISA 

European Patent Office, P.B. 5818 Patentlaan 2 
i NL - 2280 HV Rijswijk 

Tel. (+31-70) 340-2040, Tx. 31 651 epo nl, 
Fax: (+31-70) 340-3016 


Authorized officer 

Diallo, 


B 





Form PCT/ISA/210 (second sheet) (July 1992) 



page 1 of 2 



INTERNATIONAL SEARCH REPORT 



In* ational Application No 



PCT/IB 02/01841 



C.(Continuation) DOCUMENTS CONSIDERED TO BE RELEVANT 


Category ° 


Citation of document, with indication .where appropriate, of the relevant passages 


Relevant to claim Mo. 


Y 


EP 1 006 484 A (MITSUBISHI ELECTRIC CORP) 
7 June 2000 (2000-06-07) 
paragraphs 

'0022! , '0033! , '0036!-'0039! , '0046! ; claim 
1; figures 4,10,11 


2-4,6,7 


Y 


US 5 839 090 A (ZORASTER STEVEN) 
17 November 1998 (1998-11-17) 
figures 4,6,7 
column 3, line 51 -column 4, line 21 
column 5, line 1 - line 10 


8-10 


A 


B0RGEF0RS G: "DISTANCE TRANSFORMATIONS IN 
DIGITAL IMAGES" 

COMPUTER VISION GRAPHICS AND IMAGE 
PROCESSING, ACADEMIC PRESS, DULUTH, MA, 
US, 

vol. 34, 1986, pages 344-371, XP000654507 
figures 22,23 
page 368, paragraph 4.3 - paragraph 4.4 


1-20 



Form PCT/ISA/210 (continuation of second sheet) (July 1992) 



page 2 of 2 



INTERNATIONAL SEARCH REPORT 



Information on patent family members 


Int ational Application No 

PCT/IB 02/01841 


Patent document 
cited in search report 


Publication 
date 


Patent family 
member(s) 


Publication 
date 



EP 1006484 A 07-06-2000 US 6262738 Bl 17-07-2001 

EP 1006484 A2 07-06-2000 

JP 3178528 B2 18-06-2001 

JP 2000172854 A 23-06-2000 



US 5839090 A 17-11-1998 AU 693394 B2 25-06-1998 

AU 1160497 A 11-06-1997 

CA 2226090 Al 29-05-1997 

EP 0862768 Al 09-09-1998 

WO 9719424 Al 29-05-1997 



Form PCT/iaA/210 (patent family annex) (July 1992) 



